Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce some megamorphism in nodes #51380

Closed
wants to merge 1 commit into from
Closed

Reduce some megamorphism in nodes #51380

wants to merge 1 commit into from

Conversation

rbuckton
Copy link
Member

@rbuckton rbuckton commented Nov 1, 2022

This is a WIP to investigate the effects some code changes would have on reducing megamorphism in the compiler.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Nov 1, 2022
@rbuckton
Copy link
Member Author

rbuckton commented Nov 1, 2022

@typescript-bot perf test faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 1, 2022

Heya @rbuckton, I've started to run the abridged perf test suite on this PR at 976cced. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..51380

Metric main 51380 Delta Best Worst
Angular - node (v16.17.1, x64)
Memory used 353,451k (± 0.02%) 353,977k (± 0.02%) +526k (+ 0.15%) 353,723k 354,131k
Parse Time 1.90s (± 0.53%) 1.90s (± 0.80%) +0.00s (+ 0.00%) 1.88s 1.95s
Bind Time 0.75s (± 0.89%) 0.74s (± 0.30%) -0.01s (- 1.20%) 0.74s 0.75s
Check Time 5.70s (± 0.46%) 5.66s (± 0.46%) -0.05s (- 0.79%) 5.59s 5.72s
Emit Time 6.10s (± 0.47%) 6.07s (± 0.67%) -0.04s (- 0.57%) 6.02s 6.22s
Total Time 14.46s (± 0.39%) 14.37s (± 0.40%) -0.09s (- 0.61%) 14.30s 14.57s
Compiler-Unions - node (v16.17.1, x64)
Memory used 198,088k (± 0.49%) 197,665k (± 0.37%) -423k (- 0.21%) 197,166k 200,621k
Parse Time 0.79s (± 1.22%) 0.78s (± 0.51%) -0.00s (- 0.51%) 0.77s 0.79s
Bind Time 0.45s (± 0.89%) 0.46s (± 0.82%) +0.00s (+ 0.89%) 0.45s 0.46s
Check Time 6.43s (± 0.56%) 6.43s (± 0.72%) +0.00s (+ 0.03%) 6.33s 6.51s
Emit Time 2.26s (± 0.85%) 2.26s (± 0.52%) -0.00s (- 0.09%) 2.23s 2.28s
Total Time 9.92s (± 0.34%) 9.92s (± 0.49%) +0.00s (+ 0.01%) 9.83s 10.01s
Monaco - node (v16.17.1, x64)
Memory used 331,211k (± 0.02%) 331,533k (± 0.02%) +322k (+ 0.10%) 331,437k 331,670k
Parse Time 1.43s (± 0.62%) 1.43s (± 0.45%) -0.00s (- 0.14%) 1.42s 1.45s
Bind Time 0.69s (± 0.84%) 0.69s (± 0.97%) -0.01s (- 0.87%) 0.67s 0.70s
Check Time 5.47s (± 0.58%) 5.42s (± 0.62%) -0.06s (- 1.02%) 5.36s 5.52s
Emit Time 3.25s (± 0.50%) 3.24s (± 0.45%) -0.01s (- 0.28%) 3.21s 3.28s
Total Time 10.85s (± 0.36%) 10.78s (± 0.30%) -0.07s (- 0.66%) 10.70s 10.86s
TFS - node (v16.17.1, x64)
Memory used 294,130k (± 0.02%) 294,338k (± 0.01%) +208k (+ 0.07%) 294,309k 294,383k
Parse Time 1.23s (± 0.89%) 1.24s (± 1.27%) +0.01s (+ 0.90%) 1.21s 1.28s
Bind Time 0.64s (± 1.37%) 0.64s (± 0.63%) -0.00s (- 0.62%) 0.63s 0.65s
Check Time 5.14s (± 0.45%) 5.10s (± 0.50%) -0.04s (- 0.88%) 5.05s 5.17s
Emit Time 3.49s (± 0.40%) 3.46s (± 0.77%) -0.03s (- 0.92%) 3.41s 3.52s
Total Time 10.50s (± 0.39%) 10.43s (± 0.42%) -0.07s (- 0.65%) 10.36s 10.55s
material-ui - node (v16.17.1, x64)
Memory used 439,323k (± 0.02%) 439,613k (± 0.02%) +290k (+ 0.07%) 439,480k 439,791k
Parse Time 1.72s (± 1.19%) 1.69s (± 0.54%) -0.03s (- 1.92%) 1.67s 1.72s
Bind Time 0.54s (± 0.63%) 0.55s (± 0.85%) +0.01s (+ 2.42%) 0.54s 0.56s
Check Time 12.44s (± 0.57%) 12.51s (± 0.71%) +0.06s (+ 0.52%) 12.36s 12.80s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.70s (± 0.46%) 14.74s (± 0.61%) +0.04s (+ 0.28%) 14.59s 15.04s
xstate - node (v16.17.1, x64)
Memory used 554,924k (± 0.01%) 555,767k (± 0.01%) +843k (+ 0.15%) 555,668k 555,883k
Parse Time 2.31s (± 0.35%) 2.32s (± 0.45%) +0.01s (+ 0.43%) 2.30s 2.35s
Bind Time 0.89s (± 2.01%) 0.89s (± 2.01%) +0.00s (+ 0.00%) 0.87s 0.94s
Check Time 1.43s (± 0.80%) 1.43s (± 0.67%) -0.01s (- 0.56%) 1.40s 1.44s
Emit Time 0.07s (± 4.37%) 0.07s (± 3.23%) +0.00s (+ 1.47%) 0.06s 0.07s
Total Time 4.70s (± 0.39%) 4.70s (± 0.28%) -0.00s (- 0.04%) 4.67s 4.73s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-126-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51380 10
Baseline main 10

Developer Information:

Download Benchmark

@fatcerberus
Copy link

Megamorphism?

@rbuckton
Copy link
Member Author

rbuckton commented Nov 2, 2022

Megamorphism?

Megamorphic is a term used by V8 to indicate an inline cache (IC) for a property is too polymorphic and can no longer be optimized to a fast property lookup.

There's a good article on V8 polymorphism here: https://mrale.ph/blog/2015/01/11/whats-up-with-monomorphism.html

@Jack-Works
Copy link
Contributor

so does it improve?

@fatcerberus
Copy link

@rbuckton Ah, thanks. I thought it was referring to something like polymorphism in the sense of, e.g. generic functions. Wouldn’t have guessed “V8 implementation details”

@rbuckton
Copy link
Member Author

rbuckton commented Nov 2, 2022

so does it improve?

It's too early to say. I am still experimenting.

@rbuckton
Copy link
Member Author

rbuckton commented Nov 2, 2022

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 2, 2022

Heya @rbuckton, I've started to run the perf test suite on this PR at 5dc93c4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51380
Metric main 51380 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 353,929k (± 0.03%) 421,002k (± 0.02%) +67,073k (+18.95%) 420,814k 421,200k
Parse Time 1.57s (± 0.93%) 1.57s (± 0.95%) -0.00s (- 0.19%) 1.53s 1.60s
Bind Time 0.61s (± 0.85%) 0.68s (± 1.38%) +0.07s (+12.15%) 0.67s 0.70s
Check Time 4.44s (± 0.82%) 4.38s (± 0.73%) -0.06s (- 1.40%) 4.33s 4.45s
Emit Time 4.96s (± 1.15%) 4.89s (± 0.77%) -0.07s (- 1.37%) 4.80s 4.96s
Total Time 11.58s (± 0.61%) 11.52s (± 0.68%) -0.06s (- 0.50%) 11.34s 11.69s
Compiler-Unions - node (v18.10.0, x64)
Memory used 199,653k (± 0.96%) 230,725k (± 0.89%) +31,072k (+15.56%) 227,727k 234,424k
Parse Time 0.60s (± 1.66%) 0.61s (± 1.06%) +0.01s (+ 0.83%) 0.59s 0.62s
Bind Time 0.36s (± 1.60%) 0.40s (± 1.23%) +0.04s (+11.57%) 0.40s 0.42s
Check Time 5.39s (± 0.91%) 5.38s (± 0.82%) -0.01s (- 0.22%) 5.28s 5.48s
Emit Time 1.83s (± 1.03%) 1.83s (± 1.18%) +0.00s (+ 0.05%) 1.77s 1.88s
Total Time 8.18s (± 0.87%) 8.21s (± 0.79%) +0.03s (+ 0.43%) 8.06s 8.38s
Monaco - node (v18.10.0, x64)
Memory used 331,763k (± 0.02%) 403,297k (± 0.02%) +71,534k (+21.56%) 403,019k 403,393k
Parse Time 1.17s (± 0.64%) 1.18s (± 0.94%) +0.02s (+ 1.46%) 1.16s 1.22s
Bind Time 0.56s (± 0.93%) 0.64s (± 0.81%) +0.08s (+13.90%) 0.62s 0.65s
Check Time 4.33s (± 0.91%) 4.25s (± 0.69%) -0.08s (- 1.87%) 4.19s 4.32s
Emit Time 2.64s (± 0.88%) 2.55s (± 1.14%) 🟩-0.09s (- 3.37%) 2.51s 2.64s
Total Time 8.70s (± 0.67%) 8.62s (± 0.73%) -0.08s (- 0.91%) 8.52s 8.76s
TFS - node (v18.10.0, x64)
Memory used 294,811k (± 0.02%) 364,769k (± 0.02%) +69,958k (+23.73%) 364,616k 364,891k
Parse Time 0.94s (± 1.16%) 0.94s (± 0.83%) +0.00s (+ 0.11%) 0.92s 0.96s
Bind Time 0.59s (± 3.56%) 0.68s (± 2.81%) +0.09s (+16.24%) 0.64s 0.71s
Check Time 4.04s (± 0.55%) 3.96s (± 0.51%) -0.08s (- 1.86%) 3.91s 4.00s
Emit Time 2.65s (± 1.38%) 2.54s (± 1.22%) 🟩-0.11s (- 4.15%) 2.48s 2.62s
Total Time 8.21s (± 0.71%) 8.13s (± 0.73%) -0.09s (- 1.06%) 8.00s 8.27s
material-ui - node (v18.10.0, x64)
Memory used 439,876k (± 0.01%) 478,468k (± 0.02%) +38,592k (+ 8.77%) 478,276k 478,587k
Parse Time 1.38s (± 1.57%) 1.37s (± 0.69%) -0.01s (- 0.36%) 1.35s 1.39s
Bind Time 0.44s (± 1.31%) 0.51s (± 1.45%) +0.07s (+16.59%) 0.49s 0.53s
Check Time 10.86s (± 0.82%) 11.02s (± 0.93%) +0.16s (+ 1.47%) 10.82s 11.20s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 12.67s (± 0.77%) 12.90s (± 0.83%) +0.23s (+ 1.81%) 12.67s 13.10s
xstate - node (v18.10.0, x64)
Memory used 557,180k (± 0.01%) 646,751k (± 0.01%) +89,572k (+16.08%) 646,566k 646,860k
Parse Time 1.92s (± 0.44%) 1.92s (± 0.52%) +0.00s (+ 0.05%) 1.90s 1.94s
Bind Time 0.68s (± 0.87%) 0.89s (± 1.68%) +0.20s (+29.34%) 0.87s 0.94s
Check Time 1.11s (± 0.95%) 1.08s (± 0.70%) 🟩-0.04s (- 3.32%) 1.06s 1.09s
Emit Time 0.06s (± 0.00%) 0.06s (± 0.00%) 0.00s ( 0.00%) 0.06s 0.06s
Total Time 3.77s (± 0.36%) 3.93s (± 0.54%) +0.16s (+ 4.25%) 3.91s 4.01s
Angular - node (v16.17.1, x64)
Memory used 353,400k (± 0.03%) 420,436k (± 0.02%) +67,035k (+18.97%) 420,314k 420,562k
Parse Time 1.90s (± 0.44%) 1.92s (± 0.72%) +0.02s (+ 0.90%) 1.89s 1.96s
Bind Time 0.75s (± 0.64%) 0.84s (± 0.87%) +0.09s (+12.18%) 0.82s 0.85s
Check Time 5.71s (± 0.63%) 5.62s (± 0.43%) -0.09s (- 1.56%) 5.55s 5.66s
Emit Time 6.08s (± 0.55%) 5.96s (± 0.67%) -0.12s (- 2.01%) 5.89s 6.06s
Total Time 14.44s (± 0.45%) 14.34s (± 0.44%) -0.11s (- 0.74%) 14.19s 14.43s
Compiler-Unions - node (v16.17.1, x64)
Memory used 197,452k (± 0.01%) 230,349k (± 0.02%) +32,897k (+16.66%) 230,257k 230,467k
Parse Time 0.78s (± 0.67%) 0.78s (± 0.61%) +0.00s (+ 0.26%) 0.77s 0.79s
Bind Time 0.46s (± 1.13%) 0.51s (± 0.94%) +0.05s (+10.46%) 0.50s 0.52s
Check Time 6.48s (± 1.04%) 6.43s (± 0.61%) -0.05s (- 0.74%) 6.29s 6.49s
Emit Time 2.29s (± 1.07%) 2.26s (± 0.97%) -0.03s (- 1.22%) 2.22s 2.32s
Total Time 10.01s (± 0.74%) 9.98s (± 0.52%) -0.03s (- 0.26%) 9.83s 10.08s
Monaco - node (v16.17.1, x64)
Memory used 331,201k (± 0.01%) 402,661k (± 0.02%) +71,460k (+21.58%) 402,483k 402,800k
Parse Time 1.44s (± 0.83%) 1.43s (± 0.82%) -0.01s (- 0.49%) 1.41s 1.46s
Bind Time 0.69s (± 0.69%) 0.77s (± 0.97%) +0.08s (+11.54%) 0.76s 0.79s
Check Time 5.47s (± 0.45%) 5.39s (± 0.58%) -0.08s (- 1.41%) 5.31s 5.45s
Emit Time 3.24s (± 0.44%) 3.12s (± 1.00%) 🟩-0.12s (- 3.64%) 3.08s 3.19s
Total Time 10.83s (± 0.39%) 10.72s (± 0.61%) -0.11s (- 1.06%) 10.62s 10.85s
TFS - node (v16.17.1, x64)
Memory used 294,110k (± 0.02%) 364,048k (± 0.02%) +69,937k (+23.78%) 363,900k 364,149k
Parse Time 1.23s (± 1.50%) 1.22s (± 1.19%) -0.00s (- 0.33%) 1.20s 1.26s
Bind Time 0.64s (± 0.69%) 0.72s (± 0.46%) +0.08s (+12.15%) 0.71s 0.73s
Check Time 5.14s (± 0.49%) 5.01s (± 0.64%) -0.14s (- 2.66%) 4.94s 5.09s
Emit Time 3.50s (± 0.46%) 3.33s (± 0.82%) 🟩-0.17s (- 4.71%) 3.27s 3.40s
Total Time 10.52s (± 0.40%) 10.29s (± 0.52%) -0.23s (- 2.18%) 10.21s 10.46s
material-ui - node (v16.17.1, x64)
Memory used 439,268k (± 0.01%) 477,870k (± 0.01%) +38,602k (+ 8.79%) 477,756k 478,026k
Parse Time 1.71s (± 1.22%) 1.73s (± 1.70%) +0.02s (+ 1.23%) 1.69s 1.81s
Bind Time 0.54s (± 0.83%) 0.61s (± 1.27%) +0.07s (+13.20%) 0.60s 0.63s
Check Time 12.42s (± 0.28%) 12.63s (± 0.70%) +0.21s (+ 1.67%) 12.47s 12.87s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.67s (± 0.29%) 14.97s (± 0.59%) +0.30s (+ 2.05%) 14.78s 15.21s
xstate - node (v16.17.1, x64)
Memory used 554,866k (± 0.01%) 644,378k (± 0.01%) +89,512k (+16.13%) 644,261k 644,573k
Parse Time 2.32s (± 0.26%) 2.32s (± 0.45%) +0.00s (+ 0.09%) 2.30s 2.34s
Bind Time 0.90s (± 2.19%) 1.06s (± 0.80%) +0.16s (+17.54%) 1.04s 1.08s
Check Time 1.44s (± 0.75%) 1.40s (± 0.53%) -0.04s (- 2.71%) 1.39s 1.42s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 4.72s (± 0.55%) 4.84s (± 0.27%) +0.11s (+ 2.43%) 4.81s 4.86s
Angular - node (v14.15.1, x64)
Memory used 347,689k (± 0.01%) 414,724k (± 0.01%) +67,036k (+19.28%) 414,673k 414,780k
Parse Time 2.09s (± 0.94%) 2.12s (± 1.18%) +0.03s (+ 1.29%) 2.08s 2.19s
Bind Time 0.81s (± 0.62%) 0.89s (± 0.55%) +0.09s (+11.06%) 0.88s 0.90s
Check Time 6.03s (± 0.59%) 5.94s (± 0.43%) -0.09s (- 1.49%) 5.86s 5.99s
Emit Time 6.35s (± 1.04%) 6.29s (± 1.18%) -0.06s (- 0.99%) 6.18s 6.53s
Total Time 15.28s (± 0.48%) 15.25s (± 0.75%) -0.04s (- 0.23%) 15.02s 15.58s
Compiler-Unions - node (v14.15.1, x64)
Memory used 191,548k (± 0.90%) 224,574k (± 0.78%) +33,026k (+17.24%) 223,347k 229,320k
Parse Time 0.86s (± 0.39%) 0.86s (± 0.79%) +0.00s (+ 0.47%) 0.85s 0.88s
Bind Time 0.49s (± 1.01%) 0.54s (± 0.67%) +0.05s (+ 9.90%) 0.54s 0.55s
Check Time 6.80s (± 0.53%) 6.72s (± 0.55%) -0.08s (- 1.22%) 6.66s 6.79s
Emit Time 2.44s (± 1.16%) 2.40s (± 1.12%) -0.04s (- 1.68%) 2.35s 2.47s
Total Time 10.61s (± 0.57%) 10.54s (± 0.42%) -0.07s (- 0.65%) 10.45s 10.65s
Monaco - node (v14.15.1, x64)
Memory used 326,543k (± 0.01%) 398,067k (± 0.00%) +71,524k (+21.90%) 398,026k 398,101k
Parse Time 1.59s (± 0.88%) 1.59s (± 0.70%) +0.01s (+ 0.44%) 1.57s 1.62s
Bind Time 0.74s (± 0.67%) 0.82s (± 0.93%) +0.08s (+10.73%) 0.81s 0.84s
Check Time 5.80s (± 0.49%) 5.64s (± 0.46%) -0.17s (- 2.86%) 5.58s 5.68s
Emit Time 3.42s (± 0.82%) 3.25s (± 0.92%) 🟩-0.17s (- 5.08%) 3.19s 3.32s
Total Time 11.55s (± 0.47%) 11.30s (± 0.50%) -0.25s (- 2.16%) 11.16s 11.43s
TFS - node (v14.15.1, x64)
Memory used 289,803k (± 0.01%) 359,722k (± 0.01%) +69,919k (+24.13%) 359,661k 359,770k
Parse Time 1.30s (± 0.86%) 1.31s (± 0.73%) +0.01s (+ 0.77%) 1.29s 1.33s
Bind Time 0.80s (± 0.93%) 0.88s (± 0.54%) +0.07s (+ 9.22%) 0.87s 0.89s
Check Time 5.44s (± 0.40%) 5.29s (± 0.39%) -0.15s (- 2.67%) 5.24s 5.33s
Emit Time 3.65s (± 0.73%) 3.44s (± 0.69%) 🟩-0.21s (- 5.68%) 3.37s 3.48s
Total Time 11.18s (± 0.39%) 10.92s (± 0.26%) -0.26s (- 2.37%) 10.85s 10.97s
material-ui - node (v14.15.1, x64)
Memory used 435,459k (± 0.00%) 473,907k (± 0.01%) +38,447k (+ 8.83%) 473,847k 474,012k
Parse Time 1.91s (± 0.44%) 1.91s (± 0.30%) -0.00s (- 0.05%) 1.90s 1.92s
Bind Time 0.59s (± 0.88%) 0.67s (± 1.00%) +0.08s (+13.24%) 0.65s 0.68s
Check Time 13.01s (± 0.68%) 13.00s (± 0.87%) -0.01s (- 0.05%) 12.80s 13.22s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.51s (± 0.58%) 15.58s (± 0.75%) +0.07s (+ 0.45%) 15.38s 15.81s
xstate - node (v14.15.1, x64)
Memory used 543,950k (± 0.00%) 633,478k (± 0.01%) +89,529k (+16.46%) 633,391k 633,558k
Parse Time 2.61s (± 0.44%) 2.62s (± 0.59%) +0.01s (+ 0.31%) 2.60s 2.66s
Bind Time 1.00s (± 1.11%) 1.10s (± 0.45%) +0.11s (+10.73%) 1.09s 1.11s
Check Time 1.53s (± 0.84%) 1.53s (± 0.70%) -0.00s (- 0.26%) 1.50s 1.54s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.22s (± 0.35%) 5.31s (± 0.35%) +0.10s (+ 1.90%) 5.28s 5.37s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-126-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51380 10
Baseline main 10

TSServer

Comparison Report - main..51380
Metric main 51380 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,072ms (± 0.71%) 1,115ms (± 0.64%) +43ms (+ 4.00%) 1,093ms 1,130ms
Req 2 - geterr 2,757ms (± 0.90%) 2,757ms (± 0.64%) -0ms (- 0.00%) 2,728ms 2,804ms
Req 3 - references 192ms (± 1.04%) 208ms (± 1.15%) +16ms (+ 8.19%) 201ms 212ms
Req 4 - navto 148ms (± 1.09%) 149ms (± 1.02%) +1ms (+ 0.41%) 145ms 153ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 44ms (± 1.26%) 47ms (± 6.08%) +3ms (+ 5.66%) 43ms 55ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,138ms (± 0.72%) 1,189ms (± 0.49%) +51ms (+ 4.49%) 1,178ms 1,204ms
Req 2 - geterr 1,617ms (± 0.45%) 1,662ms (± 0.58%) +45ms (+ 2.78%) 1,642ms 1,684ms
Req 3 - references 199ms (± 0.94%) 219ms (± 0.62%) +20ms (+ 9.78%) 216ms 223ms
Req 4 - navto 161ms (± 1.27%) 161ms (± 1.69%) -0ms (- 0.19%) 156ms 167ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 82ms (± 4.17%) 46ms (± 3.87%) 🟩-36ms (-43.69%) 44ms 51ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,627ms (± 0.84%) 1,733ms (± 0.63%) +107ms (+ 6.55%) 1,707ms 1,753ms
Req 2 - geterr 576ms (± 0.60%) 570ms (± 0.84%) -7ms (- 1.15%) 561ms 583ms
Req 3 - references 53ms (± 3.00%) 56ms (± 3.19%) +3ms (+ 4.87%) 53ms 60ms
Req 4 - navto 207ms (± 1.13%) 207ms (± 2.05%) +1ms (+ 0.29%) 202ms 222ms
Req 5 - completionInfo count 3,209 (± 0.00%) 3,209 (± 0.00%) 0 ( 0.00%) 3,209 3,209
Req 5 - completionInfo 210ms (± 1.59%) 212ms (± 1.43%) +2ms (+ 1.05%) 206ms 220ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,330ms (± 0.49%) 1,399ms (± 0.58%) +68ms (+ 5.12%) 1,382ms 1,418ms
Req 2 - geterr 3,277ms (± 0.50%) 3,348ms (± 0.76%) +71ms (+ 2.17%) 3,306ms 3,406ms
Req 3 - references 224ms (± 1.15%) 244ms (± 1.83%) +20ms (+ 9.07%) 238ms 259ms
Req 4 - navto 158ms (± 0.44%) 157ms (± 0.82%) -1ms (- 0.70%) 154ms 159ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 53ms (± 1.09%) 56ms (± 4.41%) +3ms (+ 5.44%) 53ms 65ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,405ms (± 0.55%) 1,470ms (± 0.83%) +66ms (+ 4.67%) 1,449ms 1,497ms
Req 2 - geterr 2,113ms (± 0.50%) 2,118ms (± 0.34%) +5ms (+ 0.26%) 2,106ms 2,133ms
Req 3 - references 232ms (± 0.91%) 254ms (± 0.60%) +22ms (+ 9.44%) 251ms 258ms
Req 4 - navto 169ms (± 0.66%) 168ms (± 0.83%) -1ms (- 0.77%) 166ms 172ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 53ms (± 3.58%) 85ms (± 5.06%) 🔻+33ms (+61.98%) 78ms 95ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,932ms (± 0.50%) 2,086ms (± 0.39%) +154ms (+ 7.97%) 2,071ms 2,103ms
Req 2 - geterr 735ms (± 0.60%) 718ms (± 0.54%) -16ms (- 2.22%) 713ms 728ms
Req 3 - references 61ms (± 0.77%) 66ms (± 1.65%) +5ms (+ 8.52%) 64ms 69ms
Req 4 - navto 208ms (± 0.96%) 208ms (± 0.98%) +0ms (+ 0.05%) 204ms 213ms
Req 5 - completionInfo count 3,209 (± 0.00%) 3,209 (± 0.00%) 0 ( 0.00%) 3,209 3,209
Req 5 - completionInfo 258ms (± 0.30%) 255ms (± 1.00%) -3ms (- 1.05%) 251ms 263ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,466ms (± 0.58%) 1,522ms (± 0.45%) +57ms (+ 3.85%) 1,508ms 1,533ms
Req 2 - geterr 3,553ms (± 0.29%) 3,547ms (± 0.71%) -6ms (- 0.15%) 3,496ms 3,605ms
Req 3 - references 234ms (± 0.66%) 256ms (± 0.77%) +22ms (+ 9.31%) 253ms 261ms
Req 4 - navto 174ms (± 0.66%) 170ms (± 0.78%) -4ms (- 2.13%) 167ms 174ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 56ms (± 1.67%) 58ms (± 1.56%) +1ms (+ 2.13%) 55ms 59ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,537ms (± 0.66%) 1,600ms (± 0.68%) +63ms (+ 4.11%) 1,585ms 1,626ms
Req 2 - geterr 2,335ms (± 0.56%) 2,342ms (± 0.31%) +7ms (+ 0.29%) 2,325ms 2,357ms
Req 3 - references 245ms (± 0.63%) 270ms (± 0.72%) +25ms (+10.28%) 266ms 274ms
Req 4 - navto 184ms (± 0.51%) 180ms (± 1.03%) -4ms (- 2.12%) 175ms 183ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 56ms (± 1.07%) 61ms (± 0.91%) +5ms (+ 9.35%) 60ms 62ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,162ms (± 0.47%) 2,235ms (± 0.83%) +73ms (+ 3.40%) 2,197ms 2,276ms
Req 2 - geterr 766ms (± 0.33%) 773ms (± 0.33%) +7ms (+ 0.90%) 767ms 778ms
Req 3 - references 67ms (± 2.19%) 69ms (± 0.99%) +2ms (+ 3.16%) 67ms 70ms
Req 4 - navto 233ms (± 0.64%) 230ms (± 0.37%) -3ms (- 1.29%) 228ms 232ms
Req 5 - completionInfo count 3,209 (± 0.00%) 3,209 (± 0.00%) 0 ( 0.00%) 3,209 3,209
Req 5 - completionInfo 278ms (± 1.21%) 280ms (± 1.04%) +2ms (+ 0.83%) 271ms 284ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-126-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51380 10
Baseline main 10

Startup

Comparison Report - main..51380
Metric main 51380 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 158.08ms (± 0.34%) 160.29ms (± 0.36%) +2.21ms (+ 1.40%) 156.80ms 167.18ms
tsserver-startup - node (v16.17.1, x64)
Execution time 222.95ms (± 0.33%) 225.32ms (± 0.31%) +2.37ms (+ 1.06%) 221.74ms 234.01ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 210.99ms (± 0.30%) 213.54ms (± 0.34%) +2.55ms (+ 1.21%) 210.10ms 222.35ms
typescript-startup - node (v16.17.1, x64)
Execution time 201.03ms (± 0.33%) 203.60ms (± 0.35%) +2.57ms (+ 1.28%) 199.99ms 212.59ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-126-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51380 10
Baseline main 10

Developer Information:

Download Benchmark

@rbuckton
Copy link
Member Author

rbuckton commented Nov 3, 2022

@typescript-bot perf test faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 3, 2022

Heya @rbuckton, I've started to run the abridged perf test suite on this PR at b669700. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..51380

Metric main 51380 Delta Best Worst
Angular - node (v16.17.1, x64)
Memory used 353,400k (± 0.03%) 413,136k (± 0.01%) +59,735k (+16.90%) 412,986k 413,200k
Parse Time 1.90s (± 0.44%) 1.92s (± 0.24%) +0.02s (+ 1.11%) 1.91s 1.93s
Bind Time 0.75s (± 0.64%) 0.83s (± 2.08%) +0.09s (+11.78%) 0.82s 0.90s
Check Time 5.71s (± 0.63%) 5.54s (± 0.50%) 🟩-0.17s (- 3.05%) 5.48s 5.59s
Emit Time 6.08s (± 0.55%) 5.90s (± 0.32%) 🟩-0.19s (- 3.06%) 5.86s 5.93s
Total Time 14.44s (± 0.45%) 14.19s (± 0.36%) -0.25s (- 1.76%) 14.08s 14.29s
Compiler-Unions - node (v16.17.1, x64)
Memory used 197,452k (± 0.01%) 228,965k (± 0.02%) +31,513k (+15.96%) 228,872k 229,042k
Parse Time 0.78s (± 0.67%) 0.78s (± 0.61%) +0.00s (+ 0.26%) 0.77s 0.79s
Bind Time 0.46s (± 1.13%) 0.50s (± 0.73%) +0.04s (+ 8.06%) 0.49s 0.50s
Check Time 6.48s (± 1.04%) 6.36s (± 0.64%) -0.13s (- 1.93%) 6.26s 6.47s
Emit Time 2.29s (± 1.07%) 2.23s (± 0.85%) -0.06s (- 2.53%) 2.20s 2.29s
Total Time 10.01s (± 0.74%) 9.87s (± 0.51%) -0.14s (- 1.44%) 9.76s 9.97s
Monaco - node (v16.17.1, x64)
Memory used 331,201k (± 0.01%) 392,248k (± 0.02%) +61,047k (+18.43%) 392,112k 392,361k
Parse Time 1.44s (± 0.83%) 1.43s (± 0.49%) -0.01s (- 0.63%) 1.42s 1.45s
Bind Time 0.69s (± 0.69%) 0.76s (± 0.66%) +0.06s (+ 8.95%) 0.75s 0.77s
Check Time 5.47s (± 0.45%) 5.29s (± 0.29%) 🟩-0.18s (- 3.20%) 5.26s 5.33s
Emit Time 3.24s (± 0.44%) 3.08s (± 0.80%) 🟩-0.16s (- 4.91%) 3.03s 3.15s
Total Time 10.83s (± 0.39%) 10.56s (± 0.37%) -0.28s (- 2.56%) 10.48s 10.63s
TFS - node (v16.17.1, x64)
Memory used 294,110k (± 0.02%) 359,547k (± 0.02%) +65,437k (+22.25%) 359,365k 359,633k
Parse Time 1.23s (± 1.50%) 1.23s (± 0.90%) +0.01s (+ 0.49%) 1.21s 1.26s
Bind Time 0.64s (± 0.69%) 0.72s (± 0.81%) +0.07s (+11.68%) 0.71s 0.73s
Check Time 5.14s (± 0.49%) 4.94s (± 0.37%) 🟩-0.21s (- 4.01%) 4.91s 5.00s
Emit Time 3.50s (± 0.46%) 3.32s (± 0.54%) 🟩-0.18s (- 5.20%) 3.29s 3.37s
Total Time 10.52s (± 0.40%) 10.20s (± 0.36%) -0.31s (- 2.96%) 10.15s 10.32s
material-ui - node (v16.17.1, x64)
Memory used 439,268k (± 0.01%) 470,160k (± 0.02%) +30,892k (+ 7.03%) 470,006k 470,286k
Parse Time 1.71s (± 1.22%) 1.69s (± 0.62%) -0.02s (- 1.23%) 1.67s 1.71s
Bind Time 0.54s (± 0.83%) 0.59s (± 0.94%) +0.05s (+10.04%) 0.58s 0.60s
Check Time 12.42s (± 0.28%) 12.41s (± 0.95%) -0.01s (- 0.05%) 12.21s 12.71s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.67s (± 0.29%) 14.69s (± 0.83%) +0.03s (+ 0.17%) 14.50s 15.01s
xstate - node (v16.17.1, x64)
Memory used 554,866k (± 0.01%) 628,100k (± 0.00%) +73,235k (+13.20%) 628,059k 628,156k
Parse Time 2.32s (± 0.26%) 2.31s (± 0.33%) -0.00s (- 0.04%) 2.30s 2.33s
Bind Time 0.90s (± 2.19%) 1.03s (± 2.27%) +0.13s (+14.21%) 1.00s 1.12s
Check Time 1.44s (± 0.75%) 1.40s (± 0.64%) -0.04s (- 2.99%) 1.38s 1.42s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.23%) -0.00s (- 1.43%) 0.06s 0.07s
Total Time 4.72s (± 0.55%) 4.81s (± 0.57%) +0.09s (+ 1.82%) 4.77s 4.91s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-126-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51380 10
Baseline main 10

Developer Information:

Download Benchmark

@JoostK
Copy link
Contributor

JoostK commented Nov 3, 2022

I wouldn't have suspected quite such a steep increase in memory usage (and I almost glanced over it at first because there is no red icon shown in the memory usage row).

@Jack-Works
Copy link
Contributor

that's a little strange, if you let all node has the same shape (e.g. same property set), aren't they get optimized to some "hidden class" to save memory? or is it caused by the new extra field?

@liuxingbaoyu
Copy link

I was very surprised that the memory would increase so much, just by adding a few properties.

@fatcerberus
Copy link

fatcerberus commented Nov 4, 2022

I’m thinking the memory usage goes up because V8 is keeping more object shapes and associated JIT’d code in memory. My understanding of the inline cache mechanism was that if the engine knows the shape of an object, it can optimize property lookups down to a quick typecheck + some pointer arithmetic, which means different machine code for different object shapes. So if megamorphism is detected, it adds a layer of indirection that lets it use the same machine code for multiple object shapes (though still faster than a full property lookup; something like an inside-out vtable), which reduces memory pressure at the cost of performance.

@Jack-Works
Copy link
Contributor

but here is not adding megamorphism, but reducing it. afaic the new properties added are intended to make all the Nodes have the same shape, therefore the same optimization can apply.

@liuxingbaoyu
Copy link

Maybe v8 didn't emit native code (JIT) for them before because there were too many shapes? But either way, I'm surprised for up to 20% more memory.

@JoostK
Copy link
Contributor

JoostK commented Nov 4, 2022

The increase in memory is totally expected and has nothing to do with ICs/JIT/V8; hidden classes are obtained using a class transition graph so they are nicely shared if an object has reached a certain shape using the same transition path though that graph.

The memory increase is due to how there's now dedicated objects allocated for certain pieces of data. Previously their properties were stored on one object, but setting these properties in various orders as the program is executing means that several hidden class transitions are taken, resulting in different hidden classes and therefore resulting in megamorphic accesses for property reads that have observed more than four hidden classes (2-4 is called polymorphic).

As an experiment, this PR opts to have only singular object shapes to achieve monomorphism, but this requires that all properties are assigned up front during creation of the objects. Since some properties are only used for a small subset of nodes, this PR currently experiments with storing some properties on dedicated objects to avoid having to always store them in the node. This allows for them to be allocated on-demand (which is good, as it saves memory when not needed) it does mean that once that object is needed we're now paying for the extra allocation, indirection and memory usage.

I'd recommend this article that goes into far more detail on this subject.


My intention wasn't to blow up this thread with discussions about memory usage; this PR is clearly an experiment where various approaches are attempted to see their impact and to evaluate the trade-offs.

@fatcerberus
Copy link

fatcerberus commented Nov 4, 2022

but here is not adding megamorphism, but reducing it

I know. If you re-read my comment, I speculated that the presence of megamorphism reduces memory pressure and my guess was that that was due to being able to keep more optimized code in memory (fewer de-opts) in the non-megamorphic case, but @JoostK’s explanation indeed makes more sense.

My intention wasn't to blow up this thread with discussions about memory usage

I think it’s probably acceptable to discuss memory usage in a PR that significantly increases it. 😅 Optimizing JS can seem like a black art at times, so even the failed experiments can be instructive. btw @rbuckton already linked that article earlier in the thread, which is what prompted all this speculation in the first place. 😉

@rbuckton
Copy link
Member Author

rbuckton commented Nov 4, 2022

Maybe v8 didn't emit native code (JIT) for them before because there were too many shapes? But either way, I'm surprised for up to 20% more memory.

The increase in memory is due to the additional storage space allocated for extra (an often unused property), and the various *ExtraFields properties assigned when a NodeExtraFields is allocated. Across hundreds or thousands of nodes, this adds up.

What I've done so far is validate whether improving v8 map stability would improve performance in general by reducing the polymorphism that results from various parts of the compiler conditionally attaching properties.

The best way to achieve this while reducing the overall memory overhead is to limit such extra fields to specific node subtypes, such as moving the internal symbol property off of Node and onto Declaration, and pre-allocating a symbol property on all Declaration nodes. This reduces the overhead incurred for all nodes to only those it would have been set on in the first place.

The other benefit of this is that we can change places in the checker that currently blindly read node.symbol for nodes that will never have it defined, which results in a "wrong map" deoptimization in v8.

While the end result may still incur more memory overhead, some of that overhead will be offset by producing fewer v8 maps and fewer function recompilations due to deopts, which in turn means fewer GC pauses.

In most cases, optimizing for monomorphism doesn't give great returns. Especially for the compiler since so many functions must be polymorphic and branch on node.kind. However, we have previously run into degenerate cases of polymorphism that caused significant regressions, making this a worthwhile avenue for investigation.

@liuxingbaoyu
Copy link

liuxingbaoyu commented Nov 4, 2022

Thanks for the detailed explanation!
Just to clarify, I don't think there's anything wrong with increasing memory, I'm just out of curiosity. Sorry if anyone has misunderstood.
In my opinion this is a very, very good experiment that will provide a useful reference in the v8 world of js on how to optimize a lot of small objects. (I'm actually looking forward to it so I can do something like in @babel/parser🙂)

@ehoogeveen-medweb
Copy link

FWIW, while there may be some V8-specific details here (and V8 is the relevant target for the Node ecosystem), a lot of this applies to other JIT compilers like SpiderMonkey as well:

  1. A lot of optimization opportunities for JIT compilers rely on observing the object shapes that appear at runtime, then optimistically generating code on the assumption that later executions will behave the same way. If they don't, the generated code will fall back to a lower optimization level and evaluate whether it's worth re-compiling or sticking with something slower (garbage collections may reset various things and cause the process to start over, which is fine for longer-running code).
  2. Since higher optimization levels produce increasingly specialized code and must exist alongside more generic fallbacks, successfully JIT compiling code will often use more memory. The compilation process itself can also be memory intensive. Of course, ideally the memory is well spent.

@rbuckton
Copy link
Member Author

Closing in favor of #51682

@rbuckton rbuckton closed this Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants